The present disclosure generally relates to computer networks. More particularly, the present disclosure relates to techniques for implementing and facilitating optimization of computer-based applications in live, runtime production environments using machine learning techniques.
Many modern computer-based applications are deployed as collections of virtual infrastructure. For example, an application may be deployed as a collection of one or more virtual machines where at least one virtual machine contributes some of the overall application functionality, e.g., by providing database services, or serving web content, or providing a REST API interface. Such an application may be deployed on a private cloud or using a public cloud service such as Amazon AWS, Microsoft Azure, or Google Cloud Platform.
In general, the problem of optimizing the runtime configuration of an application is a difficult one, one whose difficulty increases with the complexity of the application (e.g., the number of components, and the number of settings of these components which may vary, such as resource assignments, replica count, tuning parameters or deployment constraints). By optimizing is here meant the determination of the settings of an application which best meet performance or service level objectives for a given application which is running in a live, runtime production environment, while generally minimizing cost (or minimizing the provisioning of unutilized/underutilized resources).
For practical examination, one may distinguish two types of application optimization, here termed continuous and discrete. Continuous optimization involves the ongoing optimization of a production application under live load (which may reflect cycles of usage as well as short or long term trends), while the application itself may also change through updates to component images, or even updates to the application architecture. Discrete optimization involves optimizing an application in a fixed environment such as a test bed or staging environment where load may be generated and controlled, and where the application components are also fixed (e.g., the VM or container image from which a component is instantiated is fixed during optimization, but the component instantiation is mutable through component settings).
Historically, optimization of even a single independent component is a non-trivial and error-prone task performed manually by a person with domain specific expertise. A multi-component application has complex interactions and limiting relations among its components, making their optimization as a harmonious system extremely difficult to achieve. The use of containerized microservices exacerbates this problem by increasing the number of application components which may need to be optimized together, increasing the dimensionality of the problem space. Often times, people may make their best guess at resource assignments for application components, test and tweak these settings a few times when first deploying the application, and leave it at that. As the application changes over time, and as the load on that application changes over time, the task of optimization may likely not be revisited until there is a performance problem, or until the cost becomes an obstacle.
An appreciation for why optimization is a difficult problem follows from an assessment of the size of the problem space. For example, if an application is comprised of five components, and at least one of these components has three settings which define its runtime configuration (e.g., CPU, memory, and network bandwidth resource assignments), and at least one setting varies through a range of 20 possible values, then there are 2015 (more than 30 quintillion) different runtime configurations in this 15-dimensional problem space. The exhaustive, or bruteforce, enumeration and assessment of some or all these combinations is impractical.
Accordingly, one objective of the present disclosure is to provide one or more automated techniques for implementing continuous optimization of computer-based applications, particularly applications running in live, runtime production environments.